Learning structure and deformation modes of nonrigid objects in long image sequences

In this paper, we present an original approach for an unsupervised learning of the structure and deformation modes of 2D moving objects in long image sequences. The object representation relies on a statistical description of the deformations applied to a prototype shape. The optimal bayesian estimate of the deformation process is obtained by maximizing a non-linear joint probability distribution using stochastic and deterministic optimization techniques. The estimates obtained at time t are integrated in the deformation model as a priori knowledge for the segmen-tation at time t + 1. Deformation modes are updated on line using a Principal Component Analysis of the distorsions computed from the shapes estimated previously in the image sequence. The approach yields robust segmentations and is demonstrated on real-world image sequences showing the tracking of hands and lips undergoing complex movements.